Skip to content

Smaller DType with Arc<StructDType> #2055

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
AdamGS merged 5 commits into from
Jan 23, 2025
Merged

Smaller DType with Arc<StructDType> #2055

AdamGS merged 5 commits into from
Jan 23, 2025

Conversation

@AdamGS
Copy link
Contributor

@AdamGS AdamGS commented Jan 22, 2025

No description provided.

@AdamGS AdamGS added the benchmark Run benchmarks on this branch label Jan 22, 2025
@github-actions github-actions bot removed the benchmark Run benchmarks on this branch label Jan 22, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Jan 22, 2025

Benchmarks: random_access

Table of Results
name PR c83a08d base ee7abec ratio (PR/base) unit
random-access/vortex-tokio-local-disk 2.05731e+06 2.0574e+06 0.999958 ns
random-access/vortex-local-fs 2.48189e+06 2.4822e+06 0.999874 ns
random-access/parquet-tokio-local-disk 2.26362e+08 2.20467e+08 1.02674 ns

@github-actions
Copy link
Contributor

github-actions bot commented Jan 22, 2025

Benchmarks: datafusion

Table of Results
name PR c83a08d base ee7abec ratio (PR/base) unit
arrow/planning 944750 946100 0.998573 ns
arrow/exec 1.98278e+06 1.97353e+06 1.00469 ns
vortex-pushdown-compressed/planning 583628 577093 1.01133 ns
vortex-pushdown-compressed/exec 2.70473e+06 2.72758e+06 0.991624 ns
vortex-pushdown-uncompressed/planning 585361 585435 0.999875 ns
vortex-pushdown-uncompressed/exec 1.57419e+06 1.57173e+06 1.00157 ns
vortex-nopushdown-compressed/planning 948456 946758 1.00179 ns
vortex-nopushdown-compressed/exec 3.21031e+06 3.24769e+06 0.988491 ns
vortex-nopushdown-uncompressed/planning 959528 955033 1.00471 ns
vortex-nopushdown-uncompressed/exec 5.44815e+06 5.1997e+06 1.04778 ns

@github-actions
Copy link
Contributor

github-actions bot commented Jan 22, 2025

Benchmarks: TPC-H

Table of Results
name PR c83a08d base ee7abec ratio (PR/base) unit
tpch_q01/arrow 560067065 5.59872e+08 1.00035 ns
tpch_q01/parquet 774026584 7.69011e+08 1.00652 ns
tpch_q01/vortex-file-compressed 535406577 5.41279e+08 0.989151 ns
tpch_q02/arrow 145153625 1.85816e+08 0.781167 ns
tpch_q02/parquet 175455071 2.27122e+08 0.772516 ns
tpch_q02/vortex-file-compressed 147975535 1.75123e+08 0.844982 ns
tpch_q03/arrow 181539156 1.90359e+08 0.953669 ns
tpch_q03/parquet 387042025 3.90283e+08 0.991696 ns
tpch_q03/vortex-file-compressed 232074617 2.44301e+08 0.949952 ns
tpch_q04/arrow 179075850 1.82021e+08 0.983817 ns
tpch_q04/parquet 217485634 2.24228e+08 0.96993 ns
tpch_q04/vortex-file-compressed 171703732 1.74885e+08 0.981811 ns
tpch_q05/arrow 326723516 3.43865e+08 0.95015 ns
tpch_q05/parquet 521282562 5.2891e+08 0.985579 ns
tpch_q05/vortex-file-compressed 389932919 3.88082e+08 1.00477 ns
tpch_q06/arrow 28201074 2.70899e+07 1.04102 ns
tpch_q06/parquet 153714370 1.53338e+08 1.00246 ns
tpch_q06/vortex-file-compressed 64447783 6.48913e+07 0.993165 ns
tpch_q07/arrow 638639921 6.433e+08 0.992757 ns
tpch_q07/parquet 783462804 8.00201e+08 0.979083 ns
tpch_q07/vortex-file-compressed 650983153 6.59473e+08 0.987126 ns
tpch_q08/arrow 268763697 2.75237e+08 0.976482 ns
tpch_q08/parquet 560410514 5.62032e+08 0.997114 ns
tpch_q08/vortex-file-compressed 361570731 3.68927e+08 0.980062 ns
tpch_q09/arrow 489104097 4.95105e+08 0.98788 ns
tpch_q09/parquet 802900499 7.97499e+08 1.00677 ns
tpch_q09/vortex-file-compressed 607788044 6.06078e+08 1.00282 ns
tpch_q10/arrow 270663032 2.71213e+08 0.997972 ns
tpch_q10/parquet 523124487 5.1972e+08 1.00655 ns
tpch_q10/vortex-file-compressed 295330106 2.88033e+08 1.02534 ns
tpch_q11/arrow 145016220 1.40935e+08 1.02895 ns
tpch_q11/parquet 156966754 1.53768e+08 1.0208 ns
tpch_q11/vortex-file-compressed 137656518 1.38429e+08 0.994419 ns
tpch_q12/arrow 188972817 1.84481e+08 1.02435 ns
tpch_q12/parquet 335181953 3.31798e+08 1.0102 ns
tpch_q12/vortex-file-compressed 254113847 2.59879e+08 0.977818 ns
tpch_q13/arrow 183528382 1.68753e+08 1.08756 ns
tpch_q13/parquet 318564420 3.30882e+08 0.962774 ns
tpch_q13/vortex-file-compressed 185846983 1.86336e+08 0.997375 ns
tpch_q14/arrow 37792670 4.08805e+07 0.924467 ns
tpch_q14/parquet 234401010 2.35859e+08 0.993817 ns
tpch_q14/vortex-file-compressed 77124884 7.90196e+07 0.976022 ns
tpch_q15/arrow 69216693 7.1437e+07 0.96892 ns
tpch_q15/parquet 327804308 3.26894e+08 1.00279 ns
tpch_q15/vortex-file-compressed 129674860 1.34993e+08 0.960604 ns
tpch_q16/arrow 99662281 1.00958e+08 0.987168 ns
tpch_q16/parquet 119454233 1.19542e+08 0.999263 ns
tpch_q16/vortex-file-compressed 106136092 1.08735e+08 0.976095 ns
tpch_q17/arrow 634939919 6.37935e+08 0.995306 ns
tpch_q17/parquet 695968517 6.96754e+08 0.998872 ns
tpch_q17/vortex-file-compressed 608036561 6.21817e+08 0.977838 ns
tpch_q18/arrow 1157435177 1.20081e+09 0.963879 ns
tpch_q18/parquet 1340912560 1.41379e+09 0.948456 ns
tpch_q18/vortex-file-compressed 1192941893 1.22693e+09 0.972296 ns
tpch_q19/arrow 154361790 1.53102e+08 1.00823 ns
tpch_q19/parquet 422892745 4.22545e+08 1.00082 ns
tpch_q19/vortex-file-compressed 147422570 1.54125e+08 0.956511 ns
tpch_q20/arrow 182505474 1.90495e+08 0.958061 ns
tpch_q20/parquet 351096549 3.22983e+08 1.08704 ns
tpch_q20/vortex-file-compressed 243112761 2.18497e+08 1.11266 ns
tpch_q21/arrow 1074190804 1.02148e+09 1.0516 ns
tpch_q21/parquet 1261613267 1.15807e+09 1.08941 ns
tpch_q21/vortex-file-compressed 1077277399 1.01837e+09 1.05784 ns
tpch_q22/arrow 81457342 8.25399e+07 0.986884 ns
tpch_q22/parquet 113212477 1.13694e+08 0.995769 ns
tpch_q22/vortex-file-compressed 87970440 8.81786e+07 0.99764 ns

@AdamGS AdamGS changed the title [WIP] smaller DType with Arc<StructDType> Smaller DType with Arc<StructDType> Jan 23, 2025
@AdamGS AdamGS marked this pull request as ready for review January 23, 2025 11:58
@AdamGS AdamGS merged commit 3015888 into develop Jan 23, 2025
21 checks passed
@AdamGS AdamGS deleted the adamg/small-arc-dtype branch January 23, 2025 12:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@gatesn gatesn gatesn approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@AdamGS @gatesn